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ABSTRACT 

Optimal  control  problems  with  a vector  performance  index  and  uncer- 
tainty in  the  state  equations  are  investigated.  It  is  assumed  that  nature 
chooses  the  uncertainty,  subject  to  bounds,  to  maximize  the  performance 
index  which  the  controller  attempts  to  minimize.  Using  Pareto  optimality 
as  the  optimality  criterion,  sufficient  conditions  for  an  optimal  solution 
are  presented.  These  conditions  also  suggest  a technique  for  determining 
the  optimal  control.  The  results  are  illustrated  with  an  example. 

1 . INTRODUCTION 

In  the  standard  optimal  control  problem,  the  system  to  be  controlled  is 
modeled  by  ordinary  differential  equations  and  the  cost  is  a scalar.  In 
many  problems,  an  exact  description  of  the  system  is  not  available  and  dis- 
turbances are  present  in  the  differential  equation  model  of  the  system. 

These  disturbances  may  have  a corrosive  influence  on  the  system  performance 
if  they  are  neglected.  Another  modification  of  the  standard  problem  is 
that  multiple  criteria,  rather  than  a single,  scalar  performance  index,  may 
be  used  to  measure  the  performance  of  the  system. 

This  paper  treats  multicriteria  optimal  control  problems  where  there  is 
uncertainty  in  the  state  equations.  An  optimality  criterion  is  defined  and 
then  sufficient  conditions  for  an  optimal  control  are  derived.  In  addition, 
it  is  shown  how  these  results  may  be  used  to  determine  an  optimal  solution. 

2.  PROBLEM  FORMULATION 

Consider  a control  system  described  by  the  differential  equations 

*(t)  - f(x(t) ,u(t) ,v(t))  (1) 

lu. 

n n jjjg  control  and  v(t)e  R is  the 

is  a C^  function.  The  time  interval' 


where  x(t)e  R“  is  the  state,  u(t)e  R 
disturbance.  We  assume  that 
■f 


[t^,t^3  is  specified  as  is  the  initial  state. 


x(t^) 


The  disturbance  v(*)  is  not  known  exactly;  the  only  information  available  V 

about  v(*)  is  that  it  belongs  to  a known  set  . Rather  than  modeling  the 

system  in  a stochastic  manner,  we  assume  that  nature  chooses  the  disturbance 
in  an  antagonistic  manner.  Thus  we  assume  that  v(*)  is  chosen  to  maximize 
the  criteria  that  the  controller  is  attempting  to  minimize.  This  is  a worst 
case  design  philosophy.  Let 


M, 


U.  );u(.)e  U and  u(t)eU,  te[t^,t^]j- 


m. 


where  U is  the  set  of  piecewise  continues  functions  from  [t^ 
U is  a given  subset  of  R Similarly,  let 


,tf] 


and 


M,.  {v(.  ):v(*)e  V and  v(t)eV,  te[t^,tj]| 


m- 


where  U is  the  set  of  piecewise  continuous  functions  from  -»  R . 

For  u(‘)eM|^  and  v(*)cM2,  the  solution  of  (1),  (2)  will  be  denoted  by  x(*). 
The  t>air  [u(*),v(*)l  is  playable  if  it  generates  a solution  of  (1),  (2) 

r 


This  research  was  supported  by  AFOSR  under  Grant  AFOSR- 76-2923. 

CaSTRIBtrtnON  statement  a I 


i jLmnovd  lor  public  release: 
CUtilbalioo  Unlimited 


1 

j 


! 


„ r.  SCI®’'"' 

. .,15  tc<*-'-*‘:-;. ,.  1 ■ i'W=.' 

”,'sr;v.utV;>  '= 

At  »•  "flcr.auon  OfrK». 

lecKil«“' 


such  that  x(tj)e9  where  the  target  set  Q Is  a given  set  in  K 

For  u(*)€M^,  N(u(*))  13  the  set  of  all  v(*)eM2  such  that  Cu(*),v(*)] 

is  playable.  A control  u(*)  is  admissible  if  u(*)cM^  and  N(u(*))^  0. 

For  a playable  pair  [u(*).v(*)l.  Che  cost  is  a vector  with  two  com- 
ponents , 

Jj^(u(*)  ,v(*)  ;x(‘))  ■ gj^(x(t^))  + J L^(x(t)  ,u(t)  ,v(t))dt,  i-1,2  (3] 

^o 

Here  x(»)  denotes  the  trajectory  corresponding  to  u(«)iV(*)  starting  at 

(x  ,t  ).  (The  results  presented  below  can  easily  be  extended  to  the  case 
o o 

where  the  cost  is  an  K vector.  For  simplicity  of  presentation,  we  con- 
centrate on  the  case  K«2.)  Our  optimality  criterion  is  an  extension  of 
the  concept  of  Pareto  optimality  for  problems  where  v(*)  is  not  present. 

* 

Definition  A control  u (•)  is  Pareto  optimal  if,  and  only  if,  it  is  admis- 
sible and  for  all  admissible  u(>)  either 

sup  Jj^(u*(*),v(*);x(*))  - sup  J^(u(*),v(«);x(*)).  1-1,2 

v(*)eN(u*(‘))  v(*)eN(u(*)) 

or,  for  at  least  one  i e{l,2}, 

sup  J (u  (•),v(*);x(*))  < sup  J. (u(*),v(*);x(*)) 


vC)eN(u  (•)) 


v(‘)eN(u(*)) 


When  the  coat  has  only  one  component,  the  definition  becomes  that  of  a 
minmax  solution  while,  if  there  is  no  disturbance,  the  definition  is  the 
usual  definition  of  Pareto  optimality.  If  an  admissible  control,  Uj^(*),  is 

not  Pareto  optimal,  then  there  is  another  admissible  control,  u-C*),  such 
that 


sup  J^(u2(').v(*);x(*))  5 sup  ,v(*)  ;x(*)) . i-1,2 

v(*)eN(u2(*))  v(.)eN(Uj^(.)) 

with  the  strict  inequality  holding  for  at  least  one  1.  Thus,  if  an  admis- 
sible control  is  not  Pareto  optimal,  there  is  another  admissible  control 
which  guarantees  that  no  component  of  the  cost  vector  increases  while  at 
least  one  decreases. 

Static  versions  of  this  problem  are  treated  in  [1-3].  In  [4],  a many 
player  differential  game  problem  with  coalitions  was  studied  and  that  prob- 
lem can  be  interpreted  as  an  optimal  control  problem  with  disturbances. 
There,  however,  in  deriving  sufficient  conditions,  it  was  assumed  that  a 
related  two-player  zero-sum  game  has  a saddle  point  solution.  We  do  not 
make  that  assumption  here. 

In  the  next  section  two  sufficient  conditions  for  Pareto  optimal  con- 
trol are  presented.  These  conditions  are  not  directly  working  conditions, 
but  only  a step  in  that  direction.  In  Sec.  4,  we  use  the  second  of  these 
conditions  to  obtain  other  conditions  which  may  be  used  for  detetming 
Pareto  optimal  controls. 

3 . PRELIMINARY  RESULTS 

Sufficient  conditions  for  Pareto  optimality  will  be  obtained  in  terms 
of  the  following  two-player  zero-sum  game. 

J(u(0,v^(*),V2(‘);Xj^(‘),X2(‘))  - a^J^(u(*),Vj^(*);x^(*)) 

+ Qf2J2(u(‘)  ,V2(*)  W 
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□ □ 


x^ct)  - f(x2(t),u(t),V2(t)), 


(5) 

(6) 


Here  x^^  (t)  ,X2  (t)*R  ,Vj^(*)  iV2(*)<M2  and  admissible  u(*)  are  defined  as  in 
Sec.  2.  One  player  controls  u(")  while  the  other  controls  v^(*)  and  V2(*)« 

A 

Note  that  x^(*)  and  v^(*)  only  enter  J through  while  u(*)enters  through 

J^and  J2.  The  time  interval  [t^.t^lis  fixed  and  we  require  x^(t^),  X2(tj)€9. 

The  controls  [u  (*) .v*(*) .V2(’)l  are  a saddle  point  solution  for  this 
game  if  they  satisfy 

J(u*(-)  ,V2(*)  ;x3^(*),X2('))  s J(u*(-)  .v*(*)  .V2(*);x*(*)  ,X2(*)) 

s:  J(u(*)  .v*(*).V2(’)  ,X2(*))  (7) 

for  V,  (•)  .v-COeNCu  (•))and  for  all  u(*)that  are  playable  against  v*(*)  and 

v-C*)*  In  Theorem  3.1, we  assume  all  u(*)€M.  are  playable  against(v.  (O.v, (•)). 
* ^ ^ ^ ^ ~ 
Theorem  3.1  If  [u  (•) |V, (•) .V2(*)3  is  a saddle  point  solution  to  the  dif- 

ferential  game  (4)-(6)  for  some  > 0>  <^1"^  ^2*  ® 

Pareto  optimal  control  for  the  problem  (l)-(3). 

Proof.  The  saddle  point  inequalities  (7)  imply 

J(u*(-),v*(*),V2C);x*(*),X2(*))  “ sup  J(u*(-).v,(‘).v,(‘);x,(*).X2(*)) 

v^C),v,C).N(u*^» 

and  that  for  any  admissible  u(*) 

J(u*(-),v*C),V2^*);x*C),X2(*))  s sup  J(u(*)>Vj^(*),V2(‘);x^(*),X2(0) 

V,  (•)  iV-COcNCuC*)) 

Thus  sup  J(u  (•).Vj^(*).V2(‘);x^(O.X2(*)) 

v^(‘) ,V2(*)€N(u*(*)) 

S sup  J(u(*)  ,Vj^(*)  .V2(*)  ;xj^(‘)  ,X2(*))  (8) 

Vj^(*)  ,V2(*)eN(u(*)) 

for  all  admissible  u(*).  Since  v^(-)  and  x^(*)  only  affect 


sup  J.(u(*),v.(*).V2(*);x,(*).X2(‘))  - ) Of.  sup  J.(u(’).v.(‘);x.(*))(9) 
Vi(*),V2(-)€N(u(-)) 

From  (8),  (9)  it  follows  that 


i-l  v^(')eN(u(*)) 


2 2 

^ Qf^  sup  J^(u*(0,V^(*);Xj^(*))  ^ sup  Jj^(u(*)  .V^(*)  ;Xj^(*))  (10) 

!■!  V (•)eN(u*(‘))  i-l  v.(*)eN(u(*)) 

Since  aya2  > 0,  (10)  implies  that  u (•)  is  Pareto  optimal.  □ 

From  this  theorem,  it  is  seen  that  Pareto  optimal  solutions  can  be 
found  by  determining  open  loop  saddle  point  solutions  for  the  two-player 
zero-sum  game  (4) -(6).  As  c»2^  and  vary,  different  saddle  point  solutions 

may  be  obtained.  Each  of  these  solutions  has  the  Pareto  optimal  property. 
Techniques  for  finding  open  loop  saddle  point  solutions  can  be  found  in 
[4-8].  Unfortunately,  many  of  the  differential  games  (4)-(6)  which  arise 


froa  problems  with  a disturbance  (l)-(3)  do  not  have  saddle  point  solutions. 

They  do,  however,  have  mlnmax  solutions.  An  admissible  control  u*(<)  Is  a 
mlnmax  solution  to  the  game  (4)-(6)  1£  It  satisfies  for  all  admissible  u(*) 

A ic 

sup  J(u  (•),v^(*).V2(‘);Xi(*),X2(*)) 

.V2(*)eN(u(*) 

S sup  J(u(*),Vj^(*).V2(*);Xj^(*),X2(’)) 

V, (•),v,(*)cN(u(‘)) 

* 

Theorem  3.2  If  u (•)  Is  a mlnmax  solution  to  the  game  (4) -(6)  for  some 
®1’®2  ^ ^l"*^  *2*  u*(*)  is  3 Pareto  optimal  control  for  the  prob- 

lem (l)-(3). 

Proof.  Since  u (•)  Is  a mlnmax  solution,  it  satisfies  (8).  The  proof  then 
follows  in  exactly  the  same  manner  as  the  proof  of  Theorem  3.1.  □ 

Theorems  similar  to  Theorem  3.1  and  Theorem  3.2  were  obtained  for  prob- 
lems Involving  coalitions  In  a differential  game  [4].  There,  use  was  made 
of  Theorem  3.1  to  find  coalitlve  Pareto  optimal  controls.  Here,  we  shall 
concentrate  on  Theorem  3.2  and  use  it,  in  conjunction  with  a sufficient  con- 
dition for  mlnmax  control  [9],  to  obtain  sufficient  conditions  for  a Pareto 
optimal  solution  to  the  problem  with  disturbances  (l)-(3).  Also  a possible 
technique  for  determining  Pareto  optimal  solutions,  based  on  these  suffi- 
cient conditions,  will  be  discussed. 


4.  MAIN  RESULT 

In  [8,9]  there  are  sufficient  conditions  for  a control  to  be  a mlnmax 
control.  Based  on  these  results,  as  well  as  Theorem  3.2,  we  obtain  suffi- 
cient conditions  for  a Pareto  optimal  control  for  the  original  problem  (1)- 
(3).  First  we  need  some  definitions.  For  i«l,2,the  set  r^(u(*))  consists  of 


the  admissible  disturbances  that  are  playable  against  u(*)  and  maximize  J^. 

£j^(u(*))  “ {v(*):v(*)eN(u(*))  and 

J^(u(*)  .v(‘)  ;x('))  “ sup  Jj^(u(*)  .v(*)  ;x(‘))} 

v(*)eN(u(*)) 

i-1.2 


n ^1 

Let  w(*,*)  be  a function  from  R x R -»  R and,  for  u(*)eM^,  9^(*)  i-S 
the  function  defined  by  q^(t)  * w(x(t),u(t))  where  x(*)  is  the  solution  of 
ii(t)  - f (x(t) ,u(t) ,w(x(t) ,u(t)) ,x(t^)  ■ x^.  The  functions  w(',*)  will  be 
called  response  functions.  The  set  of  admissible  response  function  is 
Id  ■ •|^w(*  , •)  :q^(.)eN(u(*))  all  admissible  u(.)]' 

Let  Y be  a positive  Integer,  y^(*,*)elD  1*1, ..,y,  z^(*,*)elC,  i*l,..,Y* 
Consider  an  optimal  control  problem  with  cost 


V 

g.1  (oiJi(u(‘).y^(*  ,*);x^(*))  + 

1-1 


and  state  equations 

xj^(t)  - f[xj(t),u(t),y^(xj^(t),u(t))],  1-1, ..,Y  (I2a) 

x^(t)  - f[x|(t),u(t),z^(x^(t),u(t))l,  x^(t^)  - x^  i-l,..,Y 


(12b) 


The  terminal  conditions  are  x^(tj),  X2(t^)e0,  i"l, 


.Y' 


In  the  following,  we  shall  require  > 0»  (*2  > Letting  o ■ a2foii> 
(11)  can  be  rewritten  as 


r j 


i-l 


i»l 


We  now  present  a sufficient  condition  for  a control  u (•)  to  be  a min- 
max  control  for  the  problem  (4)- (6)  and  thus,  from  Theorem  3.2,  a Pareto 
optimal  control  for  the  problem  (l)-(3). 

* 

Theorem  4. 1 Let  u (•)  be  admissible.  If  there  exists 
i)  a positive  integer  y 


T 

ii)  scalars  > 0,  i«l,..,Y,  ^ 1 and  a scalar  p > 0 


i-l 


iii)  admissible  response  y^(* , •)  »z^(* , OeUJ,  i-l,.,,Y 


such  that 


a)  v!'*(*)e  (u*(*)),  V2*(*)e 

* .f  U.  ^ J J.  ^ J J .1. 


where  v^*(t)  - y^(x^*(t)  ,u*(t) ) , V2*(t)  - z^(x2*(t)  ,u*(t))  and  xj  (O.x^  (•) 
are  solutions  of 

xj(t)  - f[x^(t),u*(t),y^(xj(t),u*(t))],  x^(t^)  - x^  i-l,..,Y 

x^Ct)  - f[x^(t),u*(t),z^(x2(t),u*(t))],  x^Ct^)  - 1-1,.., Y 


★ • w w ^ 

b)  u (•)  is  an  optimal  control  for  the  problem  P(Y»0iu,i>y  (•»*), 

i * ^ 

z (*!•))  then  u (•)  is  a Pareto  optimal  control  for  the  problem  (l)-(3). 

A * 

Proof . Consider  any  admissible  u(«)*  Since  u (•)  is  a solution  of  the 
problem  P(y  ,p  .^i^.y^(  • , * ) ) . z^(*,*)) 

Y 


i-l 


^ y,^Jj^(u*(.),vj^*(');  x^*(*))  +0  ^ u.^J2 ('»*(•  ),V2*(*));  X2*(*)) 

i-l 

Y Y 

< y u^J^(Q(-);c4(*),xJc))  + 0 y 


1-1  i-l 

qj(t)  - y^(5ij(t),&(t))  i-l...,Y 


pi(t)  - Z^tij(t),{l(t))  i-l,...Y 


Since  d > 0«  > 0 i-l,..,Yi  either 


i-l,.. ,Y 
i— 1 , . . ,Y 


(13) 


and  J2(u*(*).V2*(’);x2*(*))  - J2(^(’)  .P^ (•)  ^ 

Jj^(u  '(*)>vJ*(‘);x^*(*))  < J]^(fi(*)  .q^(*)  ;xj(‘)  for  some  ie{  1 , . . .yI 


or 


or 


J2(u*(-).V2*(*);x2*(*))  < J2(^i(*),PQ(*);x2(*))  for  some  le{ 1, . . .y} (15) 


From  (a) 

sup  Jj^(u*(*).v(«);x('))  - J^(u*(-).v[*(*);xj*(*))  1-1, ...Y  (16) 

veN(u*(’)) 

sup  J2(u*(‘).v(*);x(‘))  - J2(o*(‘),V2*(‘);x2*(‘))  i-l,..,Y  (17) 

veN(u*(’)) 

Also, 

Ji(tiC),qi(*);5iJ(*))  i sup  J^(tlC).v(-);x(-))  (18) 

v(-)eN(ft(-)) 

J2(Q(-),pJ(-);j^2^*))  5 sup  J2(^i(-),v(-).x(-))  (19) 

v(*)eN(u(*)) 

it 

Using  (16)-(19),  each  of  the  three  cases  (13),  (14),  and  (15)  imply  u (•)  is 
Pareto  optimal. 

5.  DISCUSSION 

For  a given  initial  state  and  time,  one  can  proceed  as  follows  to 
obtain  Pareto  optimal  solutions.  Form  the  two-player  zero-sum  game  (4)-(6). 
For  those  values  of  and  ^2  with  > 0 , ^2  > 0 ®nd  02'“  1 for  which 

there  is  an  open  loop  saddle  point,  the  corresponding  u(*)  is,  according  to 
Theorem  3.1,  a Pareto  optimal  solution. 

If  there  does  not  exist  a saddle  point  solution,  we  must  resort  to 

Theorem  4.1.  Choose  a real  integer  y 2 and  functions  y^(*,*),  z^(*,'), 
i-l,..,Y.  Then  determine  the  optimal  control  for  the  problem 

P(Y.p.M.i>y^(*.*),z^(‘,*))  in  terms  of  0 and  i.e.  u(  • ,p  , . . ,y^)  . If 


possible,  choose  the  so  that 

Ji(u(-),y^(-,*);x^(*))  +0  J2(u(-),z^(*,*);x2(*)) 

- Ji(n(*),y^(*  ,');x^(‘))  + p J2^'^^*^>*^^‘‘‘^5x^(-)) 

for  all  1,  jeCl,..,Y}*  for  every  p > 0 for  which  this  can  be  done,  we  then 
have  a candidate  for  a Pareto  optimal  solution  of  the  problem  (l)-(3).  It 
may  then  be  possible  to  verify  that  these  candidates  are  solutions  by  using 
Theorem  4.1.  In  checking  (a)  and  (b),  of  Theorem  4.1,  standard  sufficiency 
results  for  optimal  control,  such  as  [10-14],  can  be  used.  Of  course, 

choosing  Y,  y^(*,*)  and  z^(*,*)  may  be  difficult  and  some  ingenuity  as  well 
as  trial  and  error  must  be  used. 

The  general  idea  of  the  procedure  is  to  let  take  on  all  possible 

values  in  the  interval  (0,1).  For  every  value  of  in  this  interval  we 

hope  to  get  a Pareto  optimal  solution.  If  there  is  an  open  loop  saddle 
point  solution  corresponding  to  a particular  we  use  Theorem  3.1;  if 

not,  we  find  a minmax  solution  and  use  Theorem  4.1. 

6.  EXAMPLE 

Consider  the  following  problem  with  scalar  x(«),  u(«)  and  v(«) 


Jj^(u(*),v(*);x(‘))  - x^(l),  J2(u(*)  ,v(*);x(*))  - -x(l)  (20) 

ft(t)  - u(t)v(t),  x(0)--i  (21) 


(22) 


U - {u:u^  « 1}  . V - Cv;l  S V i 2} 

First  we  form  the  two  player  differential  game  (4)-(6) 

ij^Ct)  ■ u(t)Vj^(t)  ; x^(0)  - ~ 

X2(t)  - u(t)v2(t)  ; X2(0)  - "I 

Checking  for  open  loop  saddle  solutions  as  ranges  between  0 and  1,  we 
find  that  there  are  such  solutions  for  cy^e(0,  6/7)  and  that  the  controls 
u(*)>  v^(*)  and  V2(*)  are  constant.  These  results  are  summarized  In 
Table  1. 

TABLE  1 


a 

■u(-) 

V^(-) 

02(-) 

0 < «!  S 

1 

2 

1 

1 1 

1 - 2a, 

11  < s 

1 1 

2 

1 - Ofj^ 

1 

3 ^ ®1  ^ I 

0 

O'! 

1 

0 

2(1  - a^) 

2 

< Qfl  S 3 

, 3 

“1 

2 6 

1 - y 

< Qfl  S y 

«1 

1 

2 

The  Inequality  J(ti(*)  ,V2(’))  s J(u(*)  iV2(*))was  verified  via  the 

direct  sufficient  conditions  of  [13,14]  while  the  Inequality  J(u(*) >Vj^(*) . 

(^2(*))  S »Vj^(’)  .V2(*))  was  established  using  field  type  sufficiency 

theorems  [10,11].  That  the  u(*)  part  of  the  saddle  point  solution  Is  a 
Pareto  optimal  solution  follows  from  Theorem  3.1. 

For  6/7  < < 1,  there  is  no  open  loop  saddle  point  solution  and  we 

must  resort  to  Theorem  4.2.  To  this  end,  we  consider  the  control  problem 

given  by  (11)-(12)  with  y-2,  y\-)  - 1.  y^(-)  - 1,  z’-(-)  - 2,  z^(.)  - 2, 

1 2 
0 < p < ^,  2p  + ^ and 

K(u(*),y^(*),z^(*);xj(‘).x^(*))  - U2^Cxj(l)]^+  y,2[x^(l)]^-pg,jX2(l)  -pg.2X2(l) 

k}(t)  - u(t)  , fc^(t)  - 2u(t) 

ft^(t)  - 2u(t)  , *^(t)  - 2u(t) 

xj(0)  - xj(0)  - xj(0)  -x^(0)  - -i 

Since  u (t)  > -1/3  Is  a solution  to  this  optimal  control  problem  and  since 
y^(*)c  £j^(u*(*)).  z^(’)e  'i*(’)  ^3  a Pareto  optimal 


solution  for  this  problem.  Also,  since  p ■ 0 < p <1/6,  this 

solution  corresponds  to  6/7  < < 1.  Note  that  Theorem  3.1  does  not  apply 

when  6/7  < < 1 since  there  is  no  saddle  point  solution  and  Theorem  4.1  is 

needed. 
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